Arousal and Valence Prediction in Spontaneous Emotional Speech: Felt versus Perceived Emotion

In this paper, we describe emotion recognition experiments carried out for spontaneous affective speech with the aim to compare the added value of annotation of felt emotion versus annotation of perceived emotion. Using speech material available in the TNO-GAMING corpus (a corpus containing audiovisual recordings of people playing videogames), speech-based affect recognizers were developed that can predict Arousal and Valence scalar values. Two types of recognizers were developed in parallel: one trained with felt emotion annotations (generated by the gamers themselves) and one trained with perceived/observed emotion annotations (generated by a group of observers). The experiments showed that, in speech, with the methods and features currently used, observed emotions are easier to predict than felt emotions. The results suggest that recognition performance strongly depends on how and by whom the emotion annotations are carried out. \u

Using a conversational agent for thought recording as a cognitive therapy task: Feasibility, content, and feedback

E-mental health for depression is increasingly used in clinical practice, but patient adherence suffers as therapist involvement decreases. One reason may be the low responsiveness of existing programs: especially autonomous systems are lacking in their input interpretation and feedback-giving capabilities. Here, we explore (a) to what extent a more socially intelligent and, therefore, technologically advanced solution, namely a conversational agent, is a feasible means of collecting thought record data in dialog, (b) what people write about in their thought records, (c) whether providing content-based feedback increases motivation for thought recording, a core technique of cognitive therapy that helps patients gain an understanding of how their thoughts cause their feelings. Using the crowd-sourcing platform Prolific, 308 participants with subclinical depression symptoms were recruited and split into three conditions of varying feedback richness using the minimization method of randomization. They completed two thought recording sessions with the conversational agent: one practice session with scenarios and one open session using situations from their own lives. All participants were able to complete thought records with the agent such that the thoughts could be interpreted by the machine learning algorithm, rendering the completion of thought records with the agent feasible. Participants chose interpersonal situations nearly three times as often as achievement-related situations in the open chat session. The three most common underlying schemas were the Attachment, Competence, and Global Self-evaluation schemas. No support was found for a motivational effect of providing richer feedback. In addition to our findings, we publish the dataset of thought records for interested researchers and developers

Design patterns for an interactive storytelling robot to support children's engagement and agency

In this paper we specify and validate three interaction design patterns for an interactive storytelling experience with an autonomous social robot. The patterns enable the child to make decisions about the story by talking with the robot, reenact parts of the story together with the robot, and recording self-made sound effects. The design patterns successfully support children's engagement and agency. A user study (N = 27, 8-10 y.o.) showed that children paid more attention to the robot, enjoyed the storytelling experience more, and could recall more about the story, when the design patterns were employed by the robot during storytelling. All three aspects are important features of engagement. Children felt more autonomous during storytelling with the design patterns and highly appreciated that the design patterns allowed them to express themselves more freely. Both aspects are important features of children's agency. Important lessons we have learned are that reducing points of confusion and giving the children more time to make themselves heard by the robot will improve the patterns efficiency to support engagement and agency. Allowing children to pick and choose from a diverse set of stories and interaction settings would make the storytelling experience more inclusive for a broader range of children

Exploring Effectiveness of Explanations for Appropriate Trust: Lessons from Cognitive Psychology

The rapid development of Artificial Intelligence (AI) requires developers and designers of AI systems to focus on the collaboration between humans and machines. AI explanations of system behavior and reasoning are vital for effective collaboration by fostering appropriate trust, ensuring understanding, and addressing issues of fairness and bias. However, various contextual and subjective factors can influence an AI system explanation's effectiveness. This work draws inspiration from findings in cognitive psychology to understand how effective explanations can be designed. We identify four components to which explanation designers can pay special attention: perception, semantics, intent, and user & context. We illustrate the use of these four explanation components with an example of estimating food calories by combining text with visuals, probabilities with exemplars, and intent communication with both user and context in mind. We propose that the significant challenge for effective AI explanations is an additional step between explanation generation using algorithms not producing interpretable explanations and explanation communication. We believe this extra step will benefit from carefully considering the four explanation components outlined in our work, which can positively affect the explanation's effectiveness.Comment: 2022 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX

Personal Assistants for Healthcare Treatment at Home

ABSTRACT This paper describes the research plans in the SuperAssist project, introducing personal assistants in the care of diabetes patients, assisting the patients themselves, the medical specialists looking after the patients' healthcare, and the technical specialists responsible for maintaining the health of the devices involved. The paper discusses the issues of trust and cooperation as the critical success factors within this multi-user multi-agent (MUMA) project and within the future of agent-based healthcare attempting to increase the self-help abilities of individual patients

Inclusive design: bridging theory and practice

Abstract. Large groups in society lack the necessary skills to be sufficiently self-reliant and are in need of personal assistance. These groups could be supported by information and information technology (ICT), but only if this technology is designed to fit their (cognitive) abilities. Inclusive design theory and methods have already been developed in research contexts, but there is still a gap between theory and practice. There is a need for a practical aid, that helps to create awareness of inclusive design among ICT developers, and offers easy-to-use information and tools to actually apply the methods for diverse target groups. This paper describes the first steps taken towards an inclusive design toolbox for developing ICT applications that offer cognitive support for selfreliance. Dutch ICT companies were interviewed and participated in a co-design workshop, leading to a number of initial needs, user requirements, and an on-line community, that form input for further development of the toolbox

Considering patient safety in autonomous e-mental health systems - detecting risk situations and referring patients back to human care

Background: Digital health interventions can fill gaps in mental healthcare provision. However, autonomous e-mental health (AEMH) systems also present challenges for effective risk management. To balance autonomy and safety, AEMH systems need to detect risk situations and act on these appropriately. One option is sending automatic alerts to carers, but such 'auto-referral' could lead to missed cases or false alerts. Requiring users to actively self-refer offers an alternative, but this can also be risky as it relies on their motivation to do so. This study set out with two objectives. Firstly, to develop guidelines for risk detection and auto-referral systems. Secondly, to understand how persuasive techniques, mediated by a virtual agent, can facilitate self-referral. Methods: In a formative phase, interviews with experts, alongside a literature review, were used to develop a risk detection protocol. Two referral protocols were developed - one involving auto-referral, the other motivating users to self-refer. This latter was tested via crowd-sourcing (n = 160). Participants were asked to imagine they had sleeping problems with differing severity and user stance on seeking help. They then chatted with a virtual agent, who either directly facilitated referral, tried to persuade the user, or accepted that they did not want help. After the conversation, participants rated their intention to self-refer, to chat with the agent again, and their feeling of being heard by the agent. Results: Whether the virtual agent facilitated, persuaded or accepted, influenced all of these measures. Users who were initially negative or doubtful about self-referral could be persuaded. For users who were initially positive about seeking human care, this persuasion did not affect their intentions, indicating that a simply facilitating referral without persuasion was sufficient. Conclusion: This paper presents a protocol that elucidates the steps and decisions involved in risk detection, something that is relevant for all types of AEMH systems. In the case of self-referral, our study shows that a virtual agent can increase users' intention to self-refer. Moreover, the strategy of the agent influenced the intentions of the user afterwards. This highlights the importance of a personalised approach to promote the user's access to appropriate care.Interactive Intelligenc